Skip to content

Add German/Dutch prefixes and German title/degree suffixes - #191

Merged
derek73 merged 2 commits into
masterfrom
add-german-dutch-prefixes-titles
Jul 1, 2026
Merged

Add German/Dutch prefixes and German title/degree suffixes#191
derek73 merged 2 commits into
masterfrom
add-german-dutch-prefixes-titles

Conversation

@derek73

@derek73 derek73 commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #18 (Thomas Bachem's 58-test gist of German/Dutch names and international degrees, open for 11 years).

  • Adds prefixes: aan, aen, auf, dem, freiherr, freiherrin, heer, het, op, te, tho, thoe, vande, vd.
  • Adds titles/suffixes: Dipl.-Ing., FH-Prof., Gräfin, Me., PD, Priv.-Doz., RA, Univ.Prof., WP, ba, bsc, meng, stb, MdB/MdL/MdEP/MdA/MdHB/MdBB.
  • Fixes join_on_conjunctions() to register a conjunction-merged piece (e.g. von + und + zu) as a prefix too, mirroring the existing title-handling, so multi-word prefix chains like German "von und zu" bridge correctly into the last name instead of getting stranded in the middle name.

This takes the gist's suite from 21/68 passing to 46/68.

Deliberately not included, with reasoning verified by test:

  • to, in, an, then, ten as global prefixes — these are common Korean/Vietnamese given-name syllables in the middle-token position (e.g. Park In Hwan), and adding them regresses a currently-correct parse for those names, not just an ambiguous case.
  • bare v as a prefix (for German "v. Kloppenheim") — collides with ordinary Western middle initials (John V. Smith breaks).
  • Several remaining gist failures assert suffix for what this library correctly parses as a leading title (e.g. Mag., RA, Dipl.-Ing.) — consistent with existing conventions for Dr./MD/PhD, so not changed.
  • Multi-token suffix/title continuations like Dr. rer. nat., LL. M., M. Sc. need new joining logic beyond config additions — out of scope here.

Test plan

  • python -m pytest tests/ — 990 passed, 4 skipped, 22 xfailed (no regressions)
  • Added 3 new regression tests in tests/test_conjunctions.py covering the join_on_conjunctions prefix-bridging fix
  • Manually verified no collision for entries shared with existing constants (vd already a suffix acronym for a different meaning, ra similarly, freiherr already a leading title)

🤖 Generated with Claude Code

derek73 and others added 2 commits July 1, 2026 02:44
Closes #18. Adds prefixes (aan, aen, auf, dem, freiherr, freiherrin,
heer, het, op, te, tho, thoe, vande, vd) and titles/suffixes (Dipl.-Ing.,
FH-Prof., Gräfin, Me., PD, Priv.-Doz., RA, Univ.Prof., WP, ba, bsc, meng,
stb, MdB/MdL/MdEP/MdA/MdHB/MdBB) that don't collide with existing
English-language parsing.

Also fixes join_on_conjunctions() to register a conjunction-merged piece
(e.g. "von" + "und" + "zu") as a prefix too, mirroring the existing
title-handling, so multi-word prefix chains like German "von und zu"
bridge correctly into the last name instead of getting stranded in the
middle name.

Deliberately left out short, high-frequency English words (to, in, an,
then, ten) that collide with common Korean/Vietnamese given-name
syllables in the middle-token position, and bare "v" as a prefix, which
collides with ordinary Western middle initials.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Covers two gaps flagged in review: a merged piece that's registered as
both a title and a prefix ("freiherr"), and a chain with more than one
non-contiguous conjunction bridging prefixes into the last name.
@derek73 derek73 added this to the v1.3.0 milestone Jul 1, 2026
@derek73 derek73 self-assigned this Jul 1, 2026
@derek73
derek73 merged commit 8c75919 into master Jul 1, 2026
8 checks passed
@derek73
derek73 deleted the add-german-dutch-prefixes-titles branch July 1, 2026 09:54
derek73 added a commit that referenced this pull request Jul 3, 2026
…#191, #110/#112

Milestone audit against docs/release_log.rst found six merged, user-facing
changes with no corresponding entry: the prefix-join boundary fix (#179,
closes #100), the is_rootname() stale-cache fix (#166), extra_nickname_delimiters
(#190, closes #110/#112), international honorifics (#187), German nobility
titles (#188, closes #101), and German/Dutch prefixes plus degree suffixes
(#191, closes #18).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
derek73 added a commit that referenced this pull request Aug 24, 2026
The review round found the first draft shipped the inverse of the bug it
fixed. It let ANY piece open an entry, as the tail block always had --
safe there, because assign routes every tail piece to SUFFIX, which is
what `tail` means, and wrong off it, where a title piece routes to TITLE.

Two failures, one cause, neither visible to the gates that passed. The
`joined` tag is role-BLIND and the facade heals it for every role:

    "Smith, Rev. Dr."     title_list  ['Rev.','Dr.'] -> ['Rev. Dr.']
    "Smith Jr., Mr. Jr."  suffix      'Jr., Jr.'     -> 'Jr. Jr.'

The second glues a suffix backward across a comma the writer typed --
exactly what #429 exists to stop. The differential compares strings and
cannot see the first; the case table asserts the title STRING, which is
space-joined either way, and could not see it either.

Two joins that had been one, separated: WITHIN a piece the tag renders a
merged piece as one unit whatever role it holds; BETWEEN pieces it
continues an entry, and only a piece rendering into the same run may do
that. Sticky across a piece that is not in the entry, so an interleaved
title does not split its run ("Smith, MD Dr. PhD" -> 'MD PhD'); a
delimiter core still closes it.

Eight case rows and a facade test for the list views, which is the only
surface that shows the title collapse. Both regression guards verified
against a mutation copy -- they fail with the old condition restored.

Prose corrections, all measured by the reviewers:

- The round-trip claim was false AND backwards: str() of a fixed parse
  is a no-comma string, which re-parses with the comma back. master was
  the str-stable one. Struck from the release log and the case note.
- "one-word family comma" is not the condition -- there is no word-count
  gate, so "John Smith, Jr. III" moves too (1.4.0's reading), as does a
  title-led "Smith, Dr. MD PhD". Scope restated as it reads.
- The delimiter parity is #206 (021823e, "Apply suffix_delimiter only at
  suffix-consumption sites"), NOT #191, the German/Dutch vocabulary PR.
  Three code comments carried the error; corrected with it.
- The dormant-rule tell is #373's, and #426 the precedent for dropping a
  shadowed rule -- neither #424 entry mentions it.
- "boundary example" in the entry and both ledgers: the example FIRES,
  which is why the annotation came off.
- "filed rather than folded in" claimed an issue that does not exist.

C1 gains `_group.py` in `implemented:`, with the verbatim citation the
equality guard requires -- the whole-run half of the rule renders here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
derek73 added a commit that referenced this pull request Aug 24, 2026
…ssing

/simplify, four angles. Reuse and efficiency returned no changes; the
other two found real work.

The one that matters: the diff's central conjunct was UNPINNED. Dropping
`in_entry and` from the tag test left the whole suite green -- including
the guard added for exactly this -- while diverging on 1,152 of 65,725
inputs. That guard pins the sticky entry_open update, not the tag
condition. The distinguishing input needs a suffix piece FIRST and then
TWO titles, so the second title continues an entry it does not belong to:
"Smith, MD Rev. Dr." gives title_list ['Rev.','Dr.'] and the mutant gives
['Rev. Dr.'], with the title STRING identical either way. Added, and
verified to fail against the mutation copy.

The invariant this change rests on -- one comma segment is one suffix
entry -- was stated in NO rule. C1 governs routing; R1 governed word
ORDER and never mentioned separation; the only statement was a code
comment. That is a large part of why two stages disagreed about it for
years, so R1 gains the sentence and two examples, both names already in
the corpus (a third, "Smith, MD, PhD", would have moved for a #325
routing reason and been misfiled under #429).

Also from the review:

- Two case rows removed. family_comma_three_credential_entries is a
  strict kill-subset of the title-led row AND its note claimed to pin a
  latch that family_comma_title_between_credentials actually pins -- all
  three of its pieces are in the entry, so the latch never has to survive
  a gap. family_comma_title_run_does_not_join kills nothing, by its own
  note; test_facade.py holds the same input with assertions that can see
  the collapse.
- `family_comma` is redundant by invariant (segment() emits at most one
  segment for NO_COMMA; SUFFIX_COMMA short-circuits on `tail`). Kept for
  locality, now documented as unpinnable rather than left to look like
  missing coverage.
- I reintroduced #191 in my own new comment while this same PR corrects
  it elsewhere. Fixed here and at the two remaining delimiter sites.
  test_group.py:128 keeps #191 -- it is genuinely about the prefixes.
- The predicate moved below _is_suffix_piece, which it calls, restoring
  the _is_leading_title/_leading_titles pair it had split.
- The "whatever role it holds" claim is qualified: no multi-token TITLE
  piece witnesses that branch in 38,892 generated inputs.
- The docstring's regression narration trimmed to the contract sentence
  plus a pointer; the concrete account lives at the call site.

Efficiency recorded rather than optimised: _segment_holds_no_name runs
twice per family-comma parse, measured pure recomputation (408 of 506
comma-bearing literals reach both sites, 0 disagreements). Worth 1.2-2.2%
of a family-comma parse and 0% of every other. Caching needs a derived
ParseState field the sites would not otherwise share, so decisions.md
carries the number.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Additional unit tests mostly for German & Dutch names + more degrees

1 participant